OCCASIONAL PIECE Psychometric issues associated with computerised neuropsychological assessment of concussed athletes

Accepted 21 January 2003 . . . . . . . . . . . . . . . . . . . . . . . T he dual roles of neuropsychological testing in sports concussion are well established. Neuropsychological assessment may aid understanding of the brain structures and processes underlying concussion and the postconcussion syndrome. Although this is a primary goal of neuropsychologists working in sport concussion, a more immediate role lies in facilitating effective medical management of individual athletes after concussion. In this context, neuropsychological tests may aid both detection of post-concussive cognitive impairments and provide a de facto measurement of brain function to assist return to play decision making. Over the past two decades, ‘‘paper and pencil’’ neuropsychological tests have been used to aid the medical management of concussed professional athletes in many sports. The psychometric and practical limitations associated with these tests has led to the development of a number of computerised neuropsychological test batteries. 8 This brief article introduces some of the psychometric problems associated with computerised neuropsychological assessment in sports concussion. A number of issues critical to ensuring test reliability and sensitivity are discussed, with particular reference to how inappropriate test design can affect clinical decision making. A driving factor behind the rapid adoption of computerised neuropsychological testing is the assumption that computerised tests are both more reliable and more sensitive to concussion related cognitive deficits than paper and pencil tests. The assumption of enhanced reliability appears to be based mainly on manufacturers’ claims that computerisation of neuropsychological tests reduces administrator bias, standardises task administration, and allows randomised stimulus presentation and generation of many alternative forms. Although these practical advantages afforded by computerisation may provide a more uniformly administered test, uniform administration by itself does not necessarily bestow acceptable test reliability. Small statistical differences between groups of injured athletes and groups of control athletes are often cited as evidence of the sensitivity of neuropsychological tests. However, medical management decisions about concussed athletes are always made on a case by case basis. Evidence of differences between groups provides no information about the sensitivity of a test to change within individual athletes over time. The ability to detect subtle changes in a subject’s neuropsychological test performance, such as those commonly observed after concussion, is largely an issue of test reliability. Essentially, a reliable test is one that contains very small amounts of measurement error. When using a reliable test repeatedly, the clinician can be sure that any change in the measurement (test performance) reflects true change and not random variability. Whereas all measuring devices contain error, tests that measure abstract constructs such as cognition are prone to more error. The greater the error in a test, the less sensitive it will be to subtle change in individual subjects. Therefore test sensitivity and test reliability are closely related. When assessing the reliability and sensitivity of computerised neuropsychological tests for clinical and research use, potential users should inspect the psychometric properties of the test. Some important psychometric considerations that directly affect reliability, and therefore test sensitivity, are described below. If not designed with these in mind, computerised tests may be no more reliable or sensitive than the paper and pencil tests that they are rapidly replacing.